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Background / Context: 

Description of prior research and its intellectual context. 

Analogous thinking has been commonly discussed as being an inherent and distinguishing 
characteristic of human cognition (e.g., Gentner, 2010; Goldstone, Day, & Son, 2010; Holyoak, 
in press; Rittle- Johnson & Star, in press). Gentner (2003) has argued that as part of the human 
cognitive toolbox, comparison accompanied by the relational language to support it has rocketed 
humanity to the top of the natural hierarchy. “Although we are not the only animal with 
analogical ability, the difference in degree of ability is so great that it stands as a qualitative 
difference” (Gentner, 2003, p. 219). A core component of analogical thinking is the comparison 
of two cases to one another and the alignment and mapping of features from one case to another. 
Such comparisons have been shown in laboratory experiments to be a powerful learning practice 
compared to studying each example separately (e.g., Gentner, Loewenstein, Thompson, & 
Forbus, 2009; Gerjets, Scheiter, & Schuh, 2008, Experiment 2; Mundy, Honey, & Dwyer, 2009, 
Experiment 2; Richland & McDonough, 2010, Experiment 2; etc.). 

As a way of appropriating this informal, experiential learning that is at least partly driven by 
analogous thinking (e.g., categorization; Gentner & Holyoak, 1997), numerous formal learning 
situations have been designed with case comparisons to teach more traditional subject matter 
within classrooms (e.g., Gadgil & Nokes, 2009; Gentner, Loewenstein, Thompson, & Forbus, 
2009; Mason, 2004; Michael, Klee, Bransford, & Warren, 1993; Nagarajan & Hmelo-Silver, 
2006; Rittle- Johnson & Star, in press; Schwartz & Bransford, 1998; etc.). Analogous thinking in 
the form of learning tasks requiring case comparisons is arguably a form of scaffolded 
constructivism that is more than merely active (Chi, 2009) because it demands from the learner a 
level of engagement that goes beyond the learning materials in order to notice the underlying 
connections between them. The appreciation of those underlying connections potentially leads to 
learning (Gentner, 2010). In terms of the learning environments outlined by Chi, a learning task 
of case comparisons is not only active but also constructive (e.g., structural alignment, role-based 
relational processing, etc.; Gentner, 2010; Holyoak, in press, respectively) because it provides 
learners with two instances and asks them to construct their own understanding that serves to 
bridge the cases in the most consistent way. 

Within the case comparisons literature, there is a great variety in the ways in which cases are 
instantiated and tested both in laboratory and classroom settings. Some of these variables can be 
considered process variables because they are related to the learners’ processions through 
learning tasks. Process variables include the following: the qualities of cases, the types of 
instructions that introduce learners to the tasks, the scaffolds in place to assist learners, etc. Other 
variables can be considered generalizability variables: the domain of the subject matter (e.g., 
math, science, etc.), the age and experience levels of participants, etc. Thus, when considering 
the efficacy of case comparisons and the variety of methodologies that have previously 
investigated them, a meta-analysis seemed appropriate to get a better of sense of where the field 
is, how well our model of this learning process (see Practice section below) fits the patterns 
observed, and how future investigations could be designed best to answer remaining questions. 
Recently, Rittle- Johnson and Star (in press) requested a synthesis across studies in hopes of 
reaching a more comprehensive understanding of how comparisons aid learning and it seems that 
we are the first to attempt to meet that request. 
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Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The current meta-analysis will focus on the following three questions: 1) Overall, are case 
comparisons a method of instruction that leads to outcomes that are superior to other forms of 
instruction? 2) To what extent are such findings generalizable to all learners in all domains and 
situations? 3) What characterizes better case comparisons learning tasks and which conditions of 
our process variables fall short? Furthermore, the current study presents a new methodology for 
investigating potential moderators through meta-analytic techniques by investigating to what 
extent general trends are maintained across levels of other correlated variables. 

Setting: 

Description of the research location. 

The sample includes both classroom-based and laboratory-based studies and setting has been 
included as a potential moderator. Please see Research Design section below. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The sample includes studies of learners of all ages and experience-levels and both age and 
experience were included as potential moderators. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

For Track 2, this may include the development and validation of a measurement instrument. 

A model of the learning process. 

A process model is shown in Figure 1. Case comparisons begin with the focused, effortful 
consideration of the cases required to detect and appreciate commonalities (Seifert, McKoon, 
Abelson, & Ratcliff, 1986). The similarities between the cases (local identity matches), the 
instructions of the task (to compare, to state similarities, etc.), or both serve to highlight the 
alignable target elements or key features which can then be aligned toward structural consistency 
(Gentner, 2010; Gentner & Markman, 1997). To maximize their alignability and minimize the 
cognitive demands of the comparison task, cases should be presented simultaneously instead of 
sequentially. The alignment of the target elements of the cases leads to two general outcomes. 1 
Firstly, it shifts attention away from extraneous elements that no longer seem important because 
they do not align with any comparable elements in the other case. Secondly, it leads either to 
schema creation (if the unifying principle/concept is novel to the learner) or to schema activation 
(if the learner has previous experience on which to build). The resultant consideration of only the 
target elements, which by this time in the process have a more abstracted form, would 
consequently reduce learners’ working memory demands and therefore cognitive load because 
only the features important to the common system shared by the cases would need to be 
maintained instead of the learner having to maintain all of the details of both cases. This 
reduction in cognitive load would allow for subsequent consideration (and encoding) of the cases 
to meet the demands of the systematicity principle (i.e., a bias to prefer deeper relations; 

Gentner, 2010). What seems uncertain is whether as a result of the studied cases being 
interpreted as parts of a unifying schema, long-term memory includes only the abstracted schema 
derived from comparisons, or includes both the abstracted schema and memory for individual 
cases. Whichever the case, the abstracted long-term memory representation (i.e., schema; 



1 Whereas Gentner (2010) distinguishes four types of learning outcomes, for the purposes of reaching a general 
process model that can include our entire sample within this meta-analysis we are going to focus on two more 
general outcomes. We concede that the four types that she has outlined do fit within our model but given the limited 
information provided within articles, it seemed best to remain conservative with two. 
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Forbus, Gentner, & Law, 1994; Hintzmann, 1986) can potentially lead learners to encode newly 
encountered cases in light of that schema (Medin & Ross, 1989) or to have greater ease in 
retrieving that schema for a transfer application (Genter, Loewenstein, Thompson, & Forbus, 
2009). 

- Please insert Figure 1 here. - 

When the learner encounters analogous test problems/cases, especially those that are part of the 
same setting (class or experiment) as the study phase, the learner’s schema for such analogs is 
activated by the presence of the abstracted target elements that were identified during the 
learning activity and are now part of the schema 2 . The alignment of the new case’s elements to 
those within the schema then facilitates a correct solution or the appropriate consideration of the 
target concept as part of the studied conceptual category. Thus, case comparisons could 
potentially lead not only to correct solutions of target problems on post-tests but also more 
generally to changes in how learners encode future analogous cases and/or related subject matter, 
as is suggested by the preparation for future learning (PFL) literature among others (e.g., Medin 
& Ross, 1989; Bransford & Schwartz, 1999). 

Research Design: 

Description of the research design. 

Table 1 lists the 15 potential moderators for which we coded. Again, some were process 
variables whereas others were generalizability variables. 

- Please insert Table 1 here. - 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Relevant studies were identified through computerized literature searches on the Web of 
Science, Psyclnfo, ERIC, and Google Scholar using terms like analogical comparisons, 
structural alignment, schematic learning , etc., from citations within other articles, and email 
correspondences with authors. The selection criterion was that studies had to test directly for 
differences between a condition employing case comparisons and a condition that involved 
sequential case study, single case study, non-analogous case study, a control/baseline group (no 
study phase), or more traditional instruction (lecture and/or problem solving). Inter-rater 
reliability on the coding of our 15 potential moderators led to an overall kappa of .80. Analyses 
were conducted both at the level of experiments (i.e., average effect sizes across measures of 
single samples of learners) and at the level of tests (i.e., each effect size of all measures entered 
individually so that potential moderators could be investigated). 

Findings / Results: 

Description of the main findings with specific details. 

Using the CMA Version 2 program (Borenstein, Hedges, Higgins, & Rothstein, 2005), a 
random effects analysis of 57 experiments (336 tests) revealed an overall medium effect size 
favoring case comparisons learning situations, d = .60 (95% Cl [.47/.72]). See Table 2. The 
effects were found to be heterogeneous across samples, <2(56) = 136.74, p < .00 1 , with 59% of 
the variability among effect sizes caused by true heterogeneity between studies, 7 2 = 59.05. 
Consequently, we were justified in violating statistical assumptions of independence to 
investigate moderators at the level of tests. In order to rule out potential publication bias, we 
calculated fail-safe Ns with alphas set to .05, two-tailed. At the levels of experiments and tests. 



2 Of course, this is only one possible explanation for how the schema derived from compared cases is activated 
during testing. 
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respectively 2,795 and 5,244 unpublished studies would be needed to alter the results so that the 
effect would no longer be statistically significant. 

- Please insert Table 2 here. - 

Of the 15 potential moderators investigated, five were found to moderate findings consistently: 
content, principle, features, objective, and lag. See Table 3. Although all five were correlated 
with other variables, those trends generally held across levels of the correlated variables with few 
exceptions. Age, setting, experience, and the type of case comparisons learning task did not 
moderate effects; all showed comparable benefits of case comparisons tasks over other types of 
instruction. 

- Please insert Table 3 here. - 

Further analyses revealed that although some levels of our potential moderators were found to 
moderate effect sizes that those trends did not hold up when we examined them across levels of 
the variables with which they were correlated. The moderators of domain, context, and 
dependent measure all fit into this group. See Table 4. Furthermore, publication rank 
(publication), OLS (other learning situation), and duration have to be considered tentatively 
because of lower power in some of their levels. 

- Please insert Table 4 and Table 5 here. - 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

Analyses indicated that the effects are largely generalizable. Although few studies investigated 
adolescent-aged learners, within this sample case comparisons seems beneficial to learners of all 
ages and of all levels of experience. The marginal difference found between the effect sizes in 
math and science do not hold up under all circumstances. In studies published in second-tier 
journals, effect sizes in math and science are functionally equivalent. Effect sizes in the math, 
science, and other domain are all equivalent when the instructional design is brief and when 
learners are familiar with the subject matter. Effects also seem consistent in both laboratory and 
classroom settings but the former tend to be brief in duration and the latter tend to be long (i.e., 
strongly correlated with duration). 

When we consider the findings for our process variables, several features of case comparisons 
tasks stand out. Learners benefit most when they are asked only to find the similarities between 
cases, when they are provided with the features of at least one of the two cases, and when the 
principle is provided after the case comparisons task. Finding only the similarities helps learners 
to align the cases (Gentner, 2010) potentially because doing so is the most focused form of case 
comparisons. Learners can easily be distracted by differences and not have the same amount of 
study time to appreciate fully the shared common system. Moreover, having the features of at 
least one case provided, potentially acts as scaffolding for the learner who might otherwise be 
uncertain as to how to align the two cases. Presenting the principle after the task helps to confirm 
the learner’s potentially tentative understanding (Holyoak, in press). In a related pattern, the 
more immediate the testing, the better performances are but it is not only the forgetting curve at 
work (Ebbinghaus, 1913). It seem that the tentative nature of learners’ abstractions also factor in 
because there is no effect of the lag in testing when we only analyzed the tasks that included the 
presentation of the principle after. Such findings support claims made within the preparation-for- 
future learning (PFL; Bransford & Schwartz 1999; Schwartz & Bransford, 1998) literature. 

Overall, case comparisons tasks seem to be a promising way to get learners to be constructive 
and consequently, to learn. 
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Appendix B. Tables and Figures 
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Table 1 

LevelsofPotential Moderators within the Coding Scheme 

Moderator Definition and/or Examples 

Publication 

Top-tier Impact factor > 2 
Second-tier Impact factor < 2 
Conference Proceedings since unpublished 
Domain 



Science 

Math 



Other 



Age 

Children 

Adolescents 

Adults 

Experience 

Little 

Familiar 

Extensive 

Setting 

Classroom 

Lab 



Content 

Conceptual 



Procedural 



Perceptual 



E.g., states of matter and energy transfer (Mason, 2004) 

E.g., schema and encoding concepts (Schwartz & Bransford, 1998, E1-E3) 
E.g., solving applied algebraic problems within different domains 
(Gerjets, Scheiter, & Schuh, 2008, E2) 

E.g., solving algebraic problems (Rittle-Johnson & Star, 2007) 

E.g., getting a bead out of a cylinder (Chen & Daehler, 1989) 

E.g., negotiating contracts (Gentner, Loewenstein, & Thompson, 2004, El) 

< 12 years-old; through 6 th grade 

Between 13 and 17 years old; from 7 th through 12 th grades 
a 18 years-old; undergraduates 

Little to no experience with subject matter 

At least some experience with the content prior or concurrent to study 
Professionals in the same or a related field 

Participation was part of a class and learners' performances impacted 
their grades 

Participation was not part of a class and learners’ performances did not 
impact their grades 

Terms, definitions, categorizations 

E.g., schema quality and the quality of a recalled personal experience 
involving a contingency contract (Gentner, Loewenstein, & Thompson, 
2004, El) 

Executing steps to a solution 

E.g., implementation of a contingency contract (Gentner, Loewenstein, & 
Thompson, 2004, El) 

Perception of relationships among stimuli 

E.g., perceiving cross-mapped objects (Markman & Gentner, 1993) 
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Moderator 

Type of Case 
Comparisons 

Prompted 



Guided 



Context 

Rich 



Minimal 



Principle 

Before 

After 

Not 

Features 

Generated 

Provided 

Objective 

Similarities 

Differences 

Both 

Duration 

Brief 

Long 



Definition and/or Examples 



Learners compared cases but were not provided with instructions / 
questions that directed their attention to structural components. 

E.g., with two objects placed side-by-side, learners were asked, "Can you 
see why these are both [jiggies]?" (Christie & Gentner, 2010, El) 

E.g., learners described similarities between two cases and rated that 
similarity on a 7-point scale (Gick & Holyoak, 1983, E4) 

Learners were asked questions or were given instructions that directed 
their attention to structural components. 

Eg., provided with two solved analogs, learners were asked to explain the 
common critical insight and parallels, and then to match five elements 
between the two (Kurtz & Loewenstein, 2007, El) 

E.g., with worked examples presented side-by-side, learners explained 
how the two solutions differed (Star & Rittle-Johnson, 2009) 

Cases contained extraneous details / information. 

E.g., videotaped sessions of formative assessments (Nagarajan & Hmelo- 
Sliver, 2006) 

E.g., textually presented scenarios resolved with contingent contracts 
(Centner, Loewenstein, Thompson, & Forbus, 2009, E2) 

Cases were created in ways that highlighted structural components and 
that only including important details. 

E.g., worked examples of multistep mathematical computations (Rittle- 
Johnson & Star, 2007) 

E.g., textual presentations of only relevant information paired with 
diagrammed organizations of those facts (Centner, Loewenstein, & 
Thompson, 2003, E2-3) 

Principle was presented before the case comparisons task 
Principle was presented after the case comparisons task 
Principle was not presented 

Learners were not provided with the corresponding features. 

Learners were provided with features for at least one of the cases. 

Learners were only asked to find similarities between cases. 

Learners were only asked to find differences between cases. 

Learners were asked to find both similarities and differences. 

Single sessions of less than one hour 

Multiple sessions and/or a session of more than one hour 
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Moderator 

Other Learning 
Situation (OLS) 
Control 
Nonanalogous 



Sequential 

Single Case 
Traditional 
Dependent 
Measure (DM) 

Near 



Far 



Definition and/or Examples 



No study phase / baseline 

Learners either simultaneously compared two cases or sequentially 
studied cases but in all conditions some of those cases were not 
analogs. 

Learners studied the same cases as the case comparisons learners but did 
so sequentially instead of simultaneously. 

Learners were presented with only one analogous case to study. 

Problem solving and/or lecture 



For conceptual content, learners had to recognize or apply studied 
concepts without adaptation. 

For procedural content, learners had to transfer a previously studied, 
intact core solution. 

For conceptual content, learners had to make inferences or draw 
analogies between studied concepts and new concepts. 

For procedural content, learners had to modify studied core solutions to 
accommodate the variables or features within the new case. 
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Table 2 



Summary ofEffect Sizes 





Cohen’s d 


95% Cl 


Z 


Q 


k 


I 2 


Fixed 

Experiments 


.45 


.38/. 53 


12.10* 


136.73* 


56 


59.05 


Tests 


.37 


.34/. 41 


23.61* 


1,048. 47* 


335 


68.05 


Random 

Experiments 

Tests 


.60 

.50 


.47 / .72 
.44 / .56 


9.29* 

16.74* 









*p < .001 
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Table 3 

Moderators Found to be Consistent Across Correlated Variables 



Moderator 


Cohen's d 


95% Cl 


Z 


Q 


k 


Content 












Conceptual 


.54 


.46/. 61 


13.48** 




210 


Procedural 


.40 


.31/. 49 


8.55** 




99 


Perceptual 


.72 


.49 / .96 


6.00** 




24 










8.89* 


2 


p < .016+ 


Procedural 


Perceptual 








Conceptual 


5.03 


2.15 








Procedural 




6.27+ 








Principle 












Before 


.23 


.13/. 33 


4.38** 




73 


After 


1.18 


.93 /1.44 


9.09** 




28 


Not 


.53 


.46/. 59 


15.80** 




232 










53.39** 


2 


p < .016+ 


After 


Not 








Before 


46.03** 


22.32** 








After 




24.03** 








Features 












Generated 


.42 


.35/. 49 


11.42** 




224 


Provided 


.66 


.57/. 75 


14.35** 




110 










16.63** 


1 


Objective 












Similarities 


.68 


.60/. 75 


17.83** 




197 


Differences 


- .19 


-.30 /-.07 


-3.06* 




6 


Both 


.28 


.20/. 36 


6.66** 




130 










154.33** 


2 


p < .016+ 


Differences 


Both 








Similarities 


146.21** 


48.60** 








Differences 




40.08** 








Lag 












Immediate 


.57 


.50/. 64 


16.50** 




254 


Same Day 


.44 


.25/. 64 


4.39** 




21 


Subsequent 


.22 


.11 / .34 


3.94** 




58 










27.32** 


2 


p < .016+ 


Same Day 


Subsequent 








Immediate 


1.55 


27.21** 








Same Day 




3.57 









*p < .05, **p < .001 
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Table 4 

Moderators that Varied Between Correlated Variables 



Moderator 


Cohen’s d 


95% Cl 


Z 


Q 


k 


Domain 












Science 


.72 


.36/1.08 


3.94** 




10 


Math 


.23 


.05/ .42 


2.44* 




9 


Other 


.66 


.50/ .81 


8.24** 




35 










12.93* 


2 


p < .016+ 


Math 


Other 








Science 


5.51 


.09 








Math 




11.46+ 








Context 












Minimal 


.34 


.26/ .43 


7.74** 




128 


Rich 


.60 


.52/ .67 


15.52** 




206 










18.84** 


1 


DM 












Near 


.52 


.46/ .58 


16.11** 




287 


Far 


.34 


.20/ .48 


4.64** 




47 










5.08* 


1 



*p < .05, **p < .001 
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Table 5 



Modemtors tjiat Suffer from Low Power 



Moderator 


Cohen’s d 


95% Cl 


Z 


Q 


k 


Age 












Children 


.94 


.48/1.39 


4.04** 




9 


Adolescents 


.34 


-.01/ .68 


1.92 




4 


Adults 


.53 


.42 / .64 


9.55** 




41 










4.27 


2 


Publication 












Top 


.59 


.41 / .76 


6.50** 




30 


Second 


.73 


.50 / .95 


6.31** 




19 


Conference 


.28 


.05/ .51 


2.38* 




5 










7.85* 


2 


p < .016+ 


Second 


Conference 








Top 


.92 


4.33 








Second 




7.43+ 








OLS 












Control 


.69 


.58/ .80 


12.57** 




63 


Nonanalogous 


.82 


.55 / 1.08 


6.06** 




23 


Sequential 


.37 


.31 / .44 


11.11** 




201 


Single Case 


.92 


.57/1.28 


5.11** 




16 


Traditional 


.49 


.24/ .74 


3.87** 




29 










37.59** 


4 


p < .005 tt 


Non 


Sequ 


Single 


Trad 




Control 


.79 


24.60++ 


1.56 


1.98 




Non 




10.35++ 


.23 


3.06 




Sequential 






9.08++ 


.87 




Single 








3.81 




Duration 












Brief 


.58 


.43/ .72 


7.76** 




39 


Long 


.66 


.40/ .91 


5.00** 




16 










.27 


1 



*p < .05, ** p < .001 
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Instruction to compare cases 




Figure 1. A model of the process of case comparisons learning tasks. 
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